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Summary.  He  make  two  points  about  the  number,  B,  of  bootstrap  simulations 

needed  to  construct  a  percentile-t  confidence  interval  based  on  an  n-sample 

from  a  continuous  distribution:  (1)  The  bootstrap's  reduction  of  error  of 
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coverage  probability,  from  0(n  '  )  to  0(n  ),  is  available  uniformly  in  R, 

nrovided  nominal  coverage  probability  is  a  multiple  of  (B+l)~^.  In  fact,  this 
improvement  is  available  even  if  the  number  of  simulations  is  held  fixed  as  n 
increases,  (ii)  In  a  large  sample,  the  simulated  statistic  values  behave  like 
random  observations  from  a  continuous  distribution,  unless  B  increases  faster 
than  any  power  of  sample  size.  Only  if  B  increases  exponentially  quickly  is 
there  a  detectable  effect  due  to  discreteness  of  the  bootstrap  statistic. 
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1.  Introduction 

-  -r 

The  purpose  of  this  note  is  to  make  two  points  about  the  effect  of  the 
number  of  bootstrap  simulations,  B,  on  percentile-t  bootstraD  confidence 
intervals.  The  first  point  concerns  coverage  probability:  the  second,  distance 
of  the  "simulated"  critical  point  from  the  "true"  critical  point  derived  with 

//•-  **'  <  '  f  ys 

B=<*.  In  both  cases  we  have  in  mind  applications  to  ''smooth^  statistics,  such 
as  the  Studentized  mean  of  a  sample  drawn  from  a  continuous  distribution.  He 


sh«H+  indicate^the  changes  that  have  to  be  made  if  the  distribution  of  the 


/  J  ■  <t 

/  f  .-»v 


statistic  is  not  smooth.  t  " 

To  make  our  point  about  coverage  probability,  recall  that  if  we  conduct 


B  bootstrap  simulations,  the  resulting  statistic  values  divide  the  real  line 


into  B+l  parts.  Therefore  in  principle,  confidence  intervals  whose  critical 


points  are  based  on  B  simulations  have  coverage  probabilities  close  to  nominal 
levels  b/(B+l),  for  b=l,...,B.  If  the  sample  size  is  n  and  B=°°,  then  the 
"Edgeworth  inversion"  effect  of  the  bootstrap  argument  means  that  true  coverage 
probability  of  a  confidence  interval  whose  desired  coveraae  is  a,  is  actually  a  + 
and  6n  e  sup  |6  (cx)|  =  0(n  (Hall  1984).  This  is  a  notable  improve- 

'X 
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ment  over  the  level  a  +  0(n  '  )  offered  by  traditional  methods.  Strikingly, 
this  improvement  is  available  for  any  value  of  B,  even  for  fixed  B.  In  fact, 
if  5*  is  the  worst  possible  error  between  true  coverage  probability  and 
nominal  coverage  probability  when  only  B  simulations  are  used,  then  6*  <_  6n 

uniformly  in  B.  Therefore  the  worst  departure  of  true  coverage  probability 
fro m  nominal  coverage  probability  using  any  finite  number  of  simulations,  does 
not  exceed  the  worst  departure  using  an  infinite  number  of  simulations. 

For  example,  suppose  we  wish  to  construct  a  one-sided  90%  confidence 
interval.  The  smallest  value  of  B  we  can  use  is  B=9:  notice  that  90%  =9/(9+l). 


The  endpoint  of  the  interval  would  be  based  on  either  the  smallest  or  largest 
of  the  9  simulations,  depending  on  whether  the  interval  was  left-handed  or 
right-handed.  If  we  let  n-*»,  but  always  did  only  9  simulations,  then  the 
coveraqe  probability  of  our  interval  would  still  be  0.9  +  0(n"^).  So  even  with 
a  fixed  number  of  simulations,  we  improve  on  the  traditional  coverage  probability 
of  0.9  +  0(n‘1/2). 

This  property  forms  a  convenient  safety-net  for  the  bootstrap  algorithm: 
if  a  statistician  cannot  conduct  as  many  simulations  as  he  would  like,  he  can 
be  sure  that  he  pays  virtually  no  penalty  in  terms  of  accuracy  of  coverage 
probability.  The  only  penalty  is  in  length  of  confidence  interval  -  if  B  is 
small  then  the  true  critical  point  may  stray  from  its  limiting  value  when  B=°°, 
so  that  there  will  be  some  tendency  for  confidence  intervals  to  be  over-long. 

In  addition,  B  does  not  have  to  be  particularly  large  before  exact  coverage 
probability  agrees  with  the  theoretical  limit  as  B-*».  For  example,  if  B  equals 
sample  size  then  the  probabilities  only  disagree  at  the  level  0(n  ). 

We  shall  investigate  these  properties  in  Section  2.  As  part  of  our  study 

we  shall  give  an  explicit  formula  for  the  second-order  term  in  an  expansion  of 

coveraqe  probability  for  the  case  of  Studentized  mean.  That  formula  makes  it 

clear  that  if  the  sample  is  actually  normally  distributed,  then  even  using  a 

fixed  value  of  B,  the  coverage  probability  of  a  bootstrap  confidence  interval 

_2 

for  population  mean  differs  from  the  nominal  level  by  only  0(n  ). 

We  shall  also  investigate  the  effect  of  the  size  of  B  on  critical  points. 
Remember  that  the  distribution  of  the  bootstrap  statistic  is  discrete.  Beran 
(1984),  amonq  others,  has  Dondered  the  use  of  smoothing  techniques  to  overcome 
discreteness.  In  Section  3  we  shall  show  that  under  a  very  weak  smoothness 
assumption,  even  weaker  than  continuity,  the  distribution  of  the  simulated 


bootstrap  statistic  behaves  like  a  continuous  distribution  with  a  density 
uniformly  close  to  the  standard  normal  density.  In  fact,  the  error  in  this 
continuous  approximation  to  the  discrete  bootstrap  distribution  is  of  order 
n”A  for  all  X>0.  Ue  shall  show  that  the  number  of  bootstrap  simulations,  B,  has  to 
be  an  exponentially  large  function  of  sample  size  before  the  discreteness  of 
the  bootstrap  distribution  becomes  apparent. 

On  the  other  hand,  if  the  sampling  distribution  is  lattice  then  it  is  easily 

-1  /2 

seen  that  the  atoms  of  the  bootstrap  statistic  are  of  order  n  '  ,  and  then 
it  is  essential  to  smooth  the  distribution  of  the  bootstrap  statistic.  Our 
results  on  coverage  probability  have  analogues  for  lattice-valued  statistics, 

but  it  should  be  remembered  that  in  that  case,  roundinn  error  reduces  approxi- 

-1  -1/2 
mation  order  from  n  to  only  n  . 

We  shall  confine  attention  to  one-sided,  percentile-t  confidence  intervals. 

The  bias-corrected  percentile  technique  (Efron  1979)  is  suitable  only  for 
two-sided  intervals,  and  in  that  situation  traditional  methods,  percentile 
methods  and  bias-corrected  percentile  methods  all  give  coverage  probabilities 
whose  errors  are  of  order  n-"' .  There,  the  advantages  of  the  bootstrao  cannot 
be  reported  so  clearly  in  terms  of  coverage  probability. 

Related  work  includes  that  of  Beran  (1982,1984),  Singh  (1981)  and  Babu 
and  Singh  (1983),  who  have  studied  large-sample  properties  of  the  bootstrap 
alaorithm.  The  latter  two  papers  are  concerned  with  conditional  Edgeworth 
expansions.  In  Section  2  we  shall  briefly  mention  Edqeworth  expansions,  but 
those  considered  here  are  unconditional. 
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2.  Coverage  probability 

/N  A 

Let  0  =  e(X1,...,Xn)  be  a  ^-consistent  estimator  of  a  parameter  9,  based 

_i  -^2 

on  a  random  sample  x  *{X^  ,...Xn>.  Let  n  a  (X-|,...,Xn)  be  a  consistent 

A 

estimator  of  the  variance  of  9.  We  shall  consider  confidence  intervals  for  0 
based  on  the  statistic  T  =  n  '  (6-9)/a. 

Let  Y,  ,...,Yn  be  independent  and  identically  distributed,  conditional  on 
Xy  with  distribution  P(Y  =  X.|y)  =  n"\  1  <  j  <  n.  The  bootstrap  statistic 

kJ 

is  obtained  by  using  the  sample  x  =  (Y-j , . .  *Yn>  in  place  of  X.  Thus,  we  consider 
0*  2  0(Y-| , . . .  ,Yn) ,  o*  =  o(Y,...,Yn),  T*  h  n^2(0*-  9 )/a*.  We  may  work  out  the 
distribution  of  T*,  conditional  on  xs  to  arbitrary  accuracy  by  means  of 
simulation.  Thus,  we  may  define 


t  =  inf  {t:  P(T*  <  t|z)  >  a}. 


which  is  the  bootstrap  approximation  to  that  point  xa  such  that  PIT  <_  x^)  *  a. 
For  example,  the  "optimal"  but  unattainable  interval  IQ  =  [9  -  n"1^2xoa,°°) 
covers  e  with  probability  a;  the  interval  I-j  =  [9  -  n  tao,°°)  covers  9  with 
probability  a+0(n"^). 

In  practice  the  value  of  t^  is  usually  itself  estimated,  by  simulation. 
Conditional  on  x ,  let  T|,...,T*  be  independent  copies  of  T*.  Arrange  them  in 
ascending  order:  T^  j  <_  ...  <  Suppose  we  select  T|v+-|)  as  our  approxi¬ 

mation  to  tQ,  for  a  given  integer  0  £  v  £  B-l .  Let  p  =  P(T*  <  TU),  and 
conditional  on  X,  let  N  have  the  binomial  Bi(B,p)  distribution.  In  place  of 
I^,  we  would  use  the  interval  I^  =  [9  -  n  Tfv+1)  a»00)'  Conditional  on  x, 
the  chance  that  I2  covers  9  is 


P(T  <_  TJv+1j|.y)  =  P(at  most  v  out  of  T|,...,T|  are  <  T|y) 


=  P(N  <  v\x)  =  l  (!)  pj(l  -  p) 
j=0  J 


B-J 


Therefore  the  exact,  unconditional  coverage  probability  of  Ig »  is 

a(v,B)  =  l  l®)  \  uj(l  -  u)B'jdP(p  <  u).  12.1) 

j=0  J  Jo 

Had  we  been  able  to  do  an  indefinite  amount  of  simulation  ,  we  would  have 
taken  v  ~  aB  as  B-*»,  and  obtained  the  interval  1^  whose  coverage  probability  is 

lim  a(v,B)  =  P(p  £  a). 

B-x» 


Our  aim  is  to  determine  how  close  a(v,B)  is  to  its  limit,  P(p  <_a). 

Hall  (1984)  has  described  expansions  for  the  distribution  of  p  in  general 
circumstances.  Those  results  show  that  p  has  asymptotically  a  uniform  distri¬ 
bution,  and  that 

P(p  <_  a)  =  a  +  n  ^Rn(a)  (2.2) 


where  Rn  is  bounded  uniformly  in  n  >  1  and  0  <  a  <  1.  Define 

G(u)  =  l  (?)  uj(1-u)B"j. 
j*0  J 


I  hen  by  (2.1), 


a(v,B)  =  (v+1 ) (B+l )-1  +  n"1 


r  1 


G(u)  dRn(u). 


(2.3) 


We  call  (v+1 )(B+1 )"1  the  nominal  coverage  probability  of  confidence  interval 
Ig.  To  simplify  the  integral  in  (2.3),  let  t  =  x(v,v)  be  the  solution  of 
G(t)  =  v,  for  0  <  v  <  1.  Then 


A  r1  fG(u) 

G(u)dR  (u)  =  dR  (u)  dv 

Jo  n  Jo  n  Jo 


r1  rT  r1 

dv  I  dR  (u)  =  R  ( t ) dv , 
Jq  JO  n  Jo  n 


a(v,B)  =  (v+l)(B+l)"1  +  n 


and  so 


(2.4) 


It  is  clear  from  (2.4)  that  if  we  seek  a  confidence  interval  whose 
coverage  probability  is  a  multiple  of  (B+l)"\  then  the  worst  error  we  commit 

if  we  simulate  only  B  times  is  no  more  than  n"^  sup  |Rn(a)|.  In  view  of  (2.2), 

a 

this  is  the  worst  error  committed  if  we  simulate  an  infinite  number  of  times. 

In  this  sense  there  is  nothing  to  be  gained,  in  terms  of  coverage  probability, 
by  simulating  often.  If  we  are  after  a  95%  interval,  and  are  not  overly 
concerned  about  interval  length,  we  might  simulate  B=19  times  and  take  v=18. 

To  investigate  this  phenomenon  a  little  more  deeply,  we  shall  examine  an 
asymptotic  formula  for  Rn (a ) .  Here  it  is  convenient  to  concentrate  on  a  special 
case,  such  as  Studentized  mean.  There,  X  =  (X-j,...,X  }  is  a  scalar  random  sample 

2  ~  —  _i 

from  a  distribution  with  mean  y  and  variance  a  ,  and  0  =  y,  0  =  X  =  n  EX.. , 

a  =  n  E( X .  -  X)  .  The  bootstrap  argument  may  be  used  to  set  confidence 

2 

intervals  for  y  without  knowing  a  .  Techniques  developed  in  Hall  (1984),  although 
now  requiring  much  more  tedious  algebra  ,  give  us 

Rn(a)  =  ^(za)4)(za)  +  n"1/2<P2(zaH(za)  +  0(n_1) 

uniformly  in  a,  where  $  is  the  standard  normal  distribution  function,  <t>  =  $>', 

z  is  the  solution  of  $(z  )  =  a, 

a  or 

*l(z)  •  l  (f  *3  -  *4)z(l  +  2z2}, 

*2U)  =  -*3  Jg  (1  +  2z2){z4  ■  18z2  '  39) 

-  X3  {^(1  +2z2)(z4  +  4z2  -  23)  +  |^z2(4z2-l )} 

2 

+  X3X4  (1+2z2)(5z4  +  2z2  -  139)  +  fg(36z2  -  23)} 
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and  A.  is  the  standardized  j'th  cumulant;  for  example,  A-  =  E(X-y)  a  and 
J  ^ 

A^  =  E(X-y)^a’4-3.  An  outline  of  the  argument  is  given  in  Appendix  (i).  For 
simplicity,  define  Q^a)  s  ^.j(za)<})(za);  then 

P(p  £  a)  =  a  +  n  +  n  3^2Q2(a)  +  0(n  2)  (2.5) 

uniformly  in  a.  From  (2.4)  we  obtain: 

a(v,B)  =  (v+l)(B+1)-1  +  n"1 f1Q1 (x)dv  +  n_3/2  ^(-Odv  +  0(n"2)  (2.6) 

0  0 

uniformly  in  0  £  v  £  B-l  and  B  £  1.  A  little  asymptotic  analysis  based  on  the 
normal  approximation  to  the  binomial  shows  that 

t(v,v)  =  B-B'1/2  (6(1 -6))1/2zv  +  B"1  1  (l-28)(l+2z2)  +  o(B"1), 

1  -1  -2  -2 
where  8  =  (v  +  ^)B  .  This  expansion  holds  uniformly  in  values  v  e  (B  ,1-B  ). 

Substituting  into  (2.6)  and  noting  that  fz^ dv  =  0,  we  get: 

a(v,B)  =  a'  +  n"^Q]  (a1 )  +  n'3/2Q2(a')  +  Ofn'V1  +  n'2), 

where  a'  =  (v+1 )(B+1 )“^ .  This  expansion  is  virtually  identical  to  (2.5). 

If  B  is  chosen  so  that  nominal  coverage  probability  equals  a,  then  a(v,B) 
and  P(p  £  a)  agree  to  second  order  if  B  is  of  larger  order  than  square  root  of 
sample  size,  and  to  third  order  if  B  is  of  larger  order  than  sample  size. 


3.  Critical  Point 

For  the  sake  of  definiteness  we  shall  concentrate  on  the  Studentized 
mean.  We  shall  impose  Cramer’s  smoothness  condition  on  the  pair  ( X , ) : 

lim  sup  |E  exp(isX  +  itX2)|  <  1.  (3.1) 

I  S  |  + 1 1  ( -x» 

This  condition  holds  for  any  random  variable  X  whose  distribution  has  a  nontrivial 

continuous  component,  and  also  for  certain  singular  distributions. 

Our  initial  aim  is  to  investigate  the  smoothness  of  the  distribution  of  T*, 

conditional  on  x-  Of  course,  T*  has  a  discrete  distribution,  with  atoms 

determined  by  the  sample  x.  We  may  artificially  smooth  that  distribution  by 

adding  small,  continuous  errors  to  the  simulated  sample  points  Y^.  For  example, 

OP 

take  £  >  0  and  let  N1,...,Nn  be  independent  N(0,n  )  random  variables  independent 

of  x  and  Y.  (Remember:  X-  {X'^,...,Xn},  Y=  (Y^ , . . .  ,Yn>.)  Set  Z.hY^+N^  ,Z=n  ^EZ^  and 

T'  =  n1/2(Z  -  X)/(n_1EZ2  -  I2)172. 

The  presence  of  the  smooth  perturbations  means  that  conditional  on  X,  T' 
has  a  continuous  distribution  with  density  g,  say.  Given  any  X  >  0  we  may 
choose  £  so  large  that  with  probability  one, 

P(  IT*  -  T*|  >  n‘V)  =  0  (n"X)  (3.2) 

as  n-*-°°.  In  this  sense,  the  discrete  random  variable  T*  may  be  approximated  by 
a  continuous  variable  T ' ,  with  an  error  of  order  n"X  for  arbitrarily  large  X. 

At  first  sight  this  approximation  seems  spurious,  and  the  reader  is  justified 
in  being  very  skeptical.  It  seems  likely  that  the  density  of  g  will  closely 
track  the  atoms  of  the  discrete  distribution  of  T*,  and  so  be  quite  unsmooth. 

After  all,  the  continuous  approximation  is  only  supported  by  minute  perturbations 
N.j ,  which  are  shrinking  to  zero  at  a  rate  of  n  for  arbitrarily  large  £. 


However,  the  theorem  below  shows  that  the  density  q  is  actually  auite  smooth. 

In  fact,  no  matter  how  large  the  value  of  l,  g  uniformly  approximates  the 
standard  normal  density  $. 

Theorem  3.1.  Assume  condition  (3.1),  and  that  E(|X|^+e)  <  ror  gr,,.;6  f  >  0. 
Then  for  each  £  >  0, 

sup  | g ( x )  -  <j>(  x )  I  ->  0  (3.3) 

-oo<X<  00 

almost  surely  as  n-*». 

The  proof  uses  standard  techniques  of  Fourier  inversion,  and  will  be 
outlined  in  Appendix  (ii).  The  key  to  this  result  is  the  fact  that  the  order 
of  the  approximation  in  (3.2)  is  not  required  to  be  exponentially  small.  There 
exist  constants  cn  decreasing  very  rapidly  to  zero  such  that,  if  (3.2)  holds  for 
a  continuous  variable  T1  and  with  cn  replacing  n~\  then  the  approximation  at 
(3.3)  breaks  down.  In  that  case  the  density  g  does  track  the  atoms  of  T*  too 
closely. 

Theorem  3.1  implies  that  the  simulated  bootstrap  values  behave  like  values 

from  a  continuous  distribution,  provided  B  is  not  exponentially  large.  For 

example,  suppose  we  conduct  B  simulations  and  use  T^v+^  as  our  approximation 
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to  the  true  critical  point  t  .  Assume  v  is  chosen  so  that  v  =  aB  +  o(R  ) 
as  B-*°";  this  is  quite  reasonable,  since  we  would  usually  have  v  =  aB  +  0(1). 

We  shall  prove  below  that  if  B  increases  no  faster  than  n\  for  any  >  >  0,  then 
as  B  and  n-*»  the  conditional  probability  P{B  '  (T^+^  -t^)  <  x|y)  converaes 
to  the  probability  that  a  normal  variable  with  zero  mean  and  variance 
a?  ;  cy,(1  -a ) / $2 ( z^ ) ,  does  not  exceed  x.  Therefore  B  '  (T^+i  ^ -t^)  has  a 
limiting  N(0,a2)  density,  conditional  on  X  and  also  unconditionally.  This 
is  the  limiting  distribution  of  the  a'th  quantile  from  a  continuous 


distribution  whose  density  converges  uniformly  to  the  standard  normal  density. 

The  result  will  fail  if  B  increases  too  quickly,  but  B  has  to  increase  faster 
than  any  power  of  sample  size  before  the  discreteness  of  the  distribution  of 
T*  becomes  apparent.  Therefore,  provided  the  sampling  distribution  is  continuous, 
we  seldom  need  to  smooth  before  constructing  critical  points. 

-1  /2 

To  prove  the  result  stated  in  the  previous  paragraph,  let  y  =  t^+B  x 
and  q  £  P(T*  <  y|x),  and  observe  that  with  probability  one, 

P(B,/2<T?v*1)  -ta)5*U)«P<T*v+))<yU) 

B 

=  y  (b  p'd-Q)8"'3'  =  1  -  $[(v+l-Bq){Bq(l-q)}'1/2]+o  (1),  (3.4) 

j=v+l  J 

usinq  the  normal  approximation  to  the  binomial.  If  we  show  that  with  probability  one 

q  =  a  +  B"1/2x4>(za)  +  o  (B"1/Z),  (3.5) 

then  it  will  follow  that  the  right-hand  side  of  (3.4)  converges  almost  surely 

-1  /?  \ 
to  $[x'|>(zwHaO -a)}  ],  as  required.  Choose  X  >  0  so  large  that  B/n  -+0, 

and  let  T'  be  as  in  (3.2).  In  view  of  (3.2),  result  (3.5)  will  follow  if  we 

show  that  for  each  -°°<x<°°,  and  with  probability  one, 

P ( T ’  <  t  +  B"1/2x|X)  =  a  +  B'1/2x0(z  )  +  o  (B"1/2).  (3.6) 

Ot  Ot 

But  Theorem  3.1  implies  that 

P ( T '  <  ta  +  B"1/2x|*)  =  P(T‘  <  ta\X)  +  B'1/2x<t»(ta)  +  o  (B"1/2).  (3.7) 

It  also  follows  from  the  theorem,  and  from  the  definition  of  ta,  that 
p(T'  <  ta\x)  <_  P ( T '  <_  t  -  2B_1  |.Y )  +  0  (B_1) 

<  P(T  <  t  -  B"V)  +  0  (B_1/2)  <  a  +  0  (B‘1/2), 


and  likewise  P(T*  <  t  |x)  >  a  +  o  (B  ).  Therefore  P(T'  <  tjx)  =  a  +  o  (B  ) 
Result  (3.6)  is  now  immediate  from  (3.7). 


Appendix  (i):  Verification  of  (2.5) 


l he  proof  is  similar  to  Hall  (1984),  although  with  considerably  greater 
complexity.  The  only  smoothness  condition  reouired  is  (3.1).  To  identify 
functions  Q1  and  Q^,  let  T  be  the  Studentized  mean  and  let  TTi»1fI2»1T3  be  polynomials 
defined  by  the  following  inverse  Cornish-Fisher  expansion: 

P(T  <  x  +  n"^^TT^(x)  +  n’^M  +  n  ^^(x)}  =  4(x)  +  0(n  2). 

Formulae  may  be  derived  using  results  of  Geary  (1947);  for  example, 

tt2(x)  =  z{(5/72)X2  (4z2-1)  -  (1/12)X4(z2-3)  +  (l/4)(z2+l )} . 

Let  IT.  denote  the  version  of  ir.  in  which  X.  is  replaced  by  its  samnle  estimate 

J  J  J 
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X j ;  for  example,  X^  =  a"  n  Z(X..-)C)  -  3.  The  functions  Q-j  »Q2  are  obtainable 

from  the  relation 


P{T  <  zq  +  n"1/2n1(za)  +  n-1n2(za)  +  n_3/2TT3(za)}  =  P(P  <  a)  +  0(n"2) 


=  a  +  n  ^Q,(a)  +  n  3/,2Q„(a)  +  0(n  2). 


(A.  1 ) 


First  find  the  cumulants  of  the  random  variable 


S(a)  =  T  -  n‘1/2n1(za)  -  n"1n2(za)  -  n’3/2n3(za) 


to  order  n  ;  then  use  the  cumulants  to  obtain  an  Edoeworth  expansion  of 

_2 

P{S(a)  <_  x}  to  order  n  ;  and  finally  set  x  =  za,  to  obtain  formula  (2.5) 
via  (A.l). 

(l)For  easy  comparison  with  classical  literature,  we  assume  sample  variance  has 
divisor  n-1 ,  not  n.  Notice  that  (2.5)  Is  Invariant  under  changes  of  scale  of  T. 
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Appendix  ( 1  i ) :  Proof  of  Theorem  3.1. 

Without  loss  of  generality,  E(X)=0.  Let  W^  =  Z^-X,  w2=E(W2|y), 

U.  =  iW.,  W2-w2),  U  =  n"^2ZU1,  I  =  var(U^jx),  f  be  the  density  of  U 
conditional  on  X,  f  be  the  conditional  density  of  the  bivariate  normal 
distribution  with  zero  mean  and  covariance  I,  and  x,x0  be  the  characteristic 


functions  of  f,f  ,  respectively.  Notice  that 


g(x)=[  {( l+n”1//2v)(l+n‘^x2)’^}^2  w2f{wu(x,v) ,w2v}dv. 


V'/2 


(A. 2) 


where  u(x,vj  =  x{(1+n"l//2v)(l+n"^x2)"1}^2.  Define  gQ  by  (A. 2)  but  with  f 

,  a .  s . 

replacing  f.  It  is  easily  proved  that  sup  |g  -<j)|  -*•  0,  and  so  it  suffices 

,  a.s. 

to  show  sup  | g-g  I  ->■  0.  For  this,  we  may  show 

sup  (i  +  |y|2) |f(y)  -  f0(y)|  a^s'0. 


That  result  follows  by  Fourier  inversion  if  we  prove  that  for  nonnegative 

a.s. 

integer  vectors  y  =  (y^,y2)  with  y-j  A  ^2  1  2,  /|D'Ix~X0)l  ■+  0,  where  D  is 

the  differential  operator.  We  treat  only  y  -  0;  other  cases  are  similar. 

Characteristic  function  manipulations  common  to  estimates  of  rates  of 
converaence  show  that  for  some  small  n  >  0,  and  for  all  sufficiently  larqe  n, 

suo  |x  (t)  -  x„U)|ent  <  n'Vn 

|t|<nn1/2 

with  probability  one.  Therefore  it  suffices  to  prove 


a.s. 

IXn(t)|dt  -  0. 


(A. 2) 


;  t  >nn 
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Notice  that  if  N  is  standard  normal,  |E  exp(isN  +  itN  )|  is  integrable  in 
(s,t).  From  this  fact,  taking  n  >  5,  and  letting  £n  be  the  empiric  character- 
istic  function  of  the  sample  of  pairs  (X..,X..),  1  <_  i  £  n,  we  see  that  the  left 


side  of  (A. 2)  is  dominated  by 


const.  nU  sup  |£  (t,  -  2t,X,  t,)|n'5 

o  p  o  n  i  c  c. 

t,+t‘>rf 

,  .  .  a.s. 

for  all  sufficiently  large  n.  Finally  observe  that  sup  |£p  -  E£n|  -*  o,  and 

invoke  condition  (3.1). 
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